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ABSTRACT 


The  definition  of  what  data  is  transferred  from  the  SARSAT  ground 
tracking  station  to  operational  search  and  rescue  users  influences  how  well 
the  users  can  action  beacon  alert  data.  Based  on  an  evaluation  of 
operational  search  and  rescue  incident  data  collected  during  the  SARSAT 
Demonstration  and  Evaluation,  categorization  and  accuracy  studies  described 
herein  indicate  that  methodologies  can  be  developed  which  will 
significantly  improve  the  operational  actioning  of  SARSAT  generated  beacon 
alert  data.  The  results  of  these  studies  are  presented  and  recommended 
approaches  to  the  handling  of  SARSAT  data  are  given. 


RESUME 


Le  transfert  de$  donnees  depuis  la  station  de  poursuite  terrestre 
SARSAT  jusqu'aux  usagers  du  systeme  de  recherche  et  de  sauvetage 
operationnels  Influe  de  faqon  importante  sur  le  succes  qu'auront  les 
usagers  dans  1 'utilisation  des  donnees  d'alerte.  En  fonction  d'une 
evaluation  des  donnees  d'incident  en  matiere  de  recherche  et  de  sauvetage 
recueillies  lors  d  l'etape  de  la  demonstration  et  de  T evaluation  du 
systeme  SARSAT,  les  etudes  de  categorisation  et  de  precision  decrites 
ci-apres  revelent  la  possibility  d'etablir  des  methodologies  qui 
contribueront  de  faqon  Importante  a  ameliorer  l'usage  operationnel  des 
donnees  d'alerte  emanant  du  systeme  SARSAT.  Les  resultats  de  ces  etudes 
sont  presentes  et  des  methodes  sont  egalement  recommandees  sur  la  faqon 
d'utiliser  les  donnees  SARSAT. 
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1.0  INTRODUCTION 


The  transfer  of  beacon  alert  data  from  the  SARSAT  Local  User 
Terminal  (LUT)  to  the  Canadian  Mission  Control  Centre  (CMCC)  was  the 
subject  of  a  previous  study,  see  Reference  (1).  The  current  studies  expand 
on  this  initial  work,  and  using  data  collected  during  the  SARSAT 
Demonstration  and  Evaluation  (D&E),  assess  the  impact  of  employing  data 
categorization  schemes  to  support  the  actioning  by  the  Search  and  Rescue 
(SAR)  community  of  those  SAR  incidents  either  initiated  or  supported  by 
SARSAT  alert  data. 

A  detailed  review  and  analysis  of  operational  SAR  incidents 
involving  the  use  of  SARSAT  data  collected  during  the  SARSAT  D&E,  i.e. 
during  the  period  February  1983  to  June  1984,  was  carried  out  and  provided 
the  basis  for  defining  and  evaluating  these  data  categorization  schemes. 
The  SAR  incidents  necessarily  of  primary  interest  for  such  studies  were  the 
traced  beacon  transmissions.  In  these  incidents,  the  actual  location  of 
the  beacon  was  derived  and  hence  categorization  studies  could  be  supported 
by  accuracy  of  location  studies. 

The  SARSAT  operational  data  categorization  and  accuracy  studies 
are  discussed  in  terms  of  the  background  to  the  problem,  an  outline  of 
approach  and  analytical  tools  developed,  a  presentation  of  the  results  of 
the  analysis  of  the  available  operational  data,  and  a  discussion  of  these 
results  in  terms  of  procedures  for  the  operational  actioning  of  SARSAT 
data.  Summary  comments  and  recommendations  are  provided. 


2.0  BACKGROUND 


The  SARSAT  system  and  the  companion  international  COSPAS-SARSAT 
project  are  described  in  detail  in  numerous  documents,  see  Reference  (2) 
and  (3),  and  therefore,  only  a  summary  of  the  system  concept  is  presented 
as  background  to  the  current  studies.  This  is  followed  by  a  brief  review 
of  the  perceived  operational  data  handling  problems  which  initiated  the 
current  work.  With  this  background  in  place,  the  specific  objectives  of 
the  data  categorization  and  accuracy  studies  are  defined. 


2.1  SARSAT  FACILITY  OVERVIEW 


The  basic  concept  of  the  SARSAT  satellite-aided  search  and  rescue 
mission  is  illustrated  in  Figure  1.  The  signal  radiated  by  an  emergency 
beacon,  either  an  Emergency  Locator  Transmitter  (ELT)  or  an  Emergency 
Position  Indicating  Radio  Beacon  {EPIRB),  is  detected  by  a  polar-orbiting 
spacecraft  equipped  with  suitable  receivers.  Such  signals  are  then  relayed 
to  a  SARSAT  ground  tracking  station  or  Local  User  Terminal  where  the 
signals  are  processed  to  determine  the  location  of  the  ELT  or  EPIRB.  The 
fact  that  an  alert  has  been  detected,  along  with  the  location  of  the  ELT  or 
EPIRB,  is  then  relayed  via  a  Mission  Control  Centre  (MCC)  to  an  appropriate 
Rescue  Coordination  Centre  (RCC)  for  initiation  of  the  search  and  rescue 
activities. 

Doppler-positioning,  using  the  relative  motion  between  the 
spacecraft  and  the  ELT/EPIRB,  was  considered  as  a  practical  means  of 
locating  these  very  simple  beacons.  All  that  is  required  of  the  ELT/EPIRB 
is  that  it  emit  a  carrier  frequency  with  a  reasonable  stability  during  the 
period  of  mutual  ELT/EPIRB-satel 1 i te  visibility.  To  optimize 
Doppler-positioning  performance,  satellites  in  a  low-altitude  polar  orbit 
are  used.  The  low  altitude  results  in  low  ELT/EPIRB  power  requirements, 
good  Doppler-shift  characteristics  and  short  time  delays  between  successive 
passes.  The  polar  orbit  results  in  coverage  of  the  whole  earth. 

Within  the  context  of  the  current  discussion,  the  system  consists 
of  the  following  subsystems: 

.  The  first  subsystem  is  the  ELT  and  EPIRB.  These  small 
emergency  transmitters  are  designed  to  transmit  distress 
signals  in  the  121.5  and  243  MHz  bands. 

.  The  second  subsystem  is  the  spacecraft  (SARSAT  and/or  COSPAS) 
which  receive  these  signals  and  retransmit  them  at  1544.5  MHz 
to  the  ground  station  for  processing. 

.  The  third  subsystem  is  the  ground  segment  facilities  which 
consists  of  the  LUT  which  receives  the  relayed  distress 
signals,  process  them  to  derive  a  location  estimate,  the  MCC 
which  receives  these  data  and  then  distributes  them  for 
operational  actioning  by  the  SAR  users  at  the  RCC. 

The  studies  discussed  herein  focus  on  the  activities  in  the  third 
subsystem.  Specifically,  the  problem  being  addressed  is  associated  with 
the  definition  of  the  transfer  of  distress  data  from  the  LUT  to  the 
Canadian  MCC  and  the  handling  of  these  data  at  the  CMCC. 


Basic  Concept 


2.2  STATEMENT  OF  THE  PROBLEM 


At  Reference  (1),  it  was  noted  that  the  problem  existing  at  the 
operational  level  with  regard  to  SARSAT  alert  information  was  one  of  a  lack 
of  data  quantities  being  associated  with  location  estimates.  Alert  data 
generated  by  the  SARSAT  LUT  is  transferred  through  to  the  CMCC  for 
operational  actioning  with  very  little  guidance  provided  on  the  quality  of 
the  data.  The  user  is  forced  to  treat  all  SARSAT  data  in  the  same  manner. 
The  CMCC  operators  and  the  RCC  controllers  cannot  measure  their  response  to 
the  alert  data  and  act  accordingly. 

The  SARSAT  system  can  produce  very  accurate  estimates  of  location 
of  transmitting  ELT/EPIRBs,  but,  it  can  also  produce  poor  estimates.  This 
range  in  capability  is  due  to  a  large  number  of  factors,  the  primary  ones 
being  the  characteristics  of  the  beacon  and  the  beacon-to-satel lite 
geometry.  The  problem,  therefore,  is  to  identify  the  intrinsic  quality  of 
the  data  given  the  conditions  under  which  it  was  derived. 

An  approach  was  developed  at  Reference  (1)  to  classify  incoming 
SARSAT  alert  data  and  procedures  were  suggested  for  actioning  these  data. 
Furthermore,  the  potential  impact  of  merging  data  from  pass-to-pass  using 
Kalman  Filter  techniques  was  illustrated.  A  LUT-to-CMCC  emulator  was 
developed  and  tested  over  a  sample  period  using  operational  data.  However, 
this  work  was  restricted  in  scope  because  the  locations  of  the  ELT/EPIRB 
detections  used  in  the  study  were  not  known.  In  other  words,  while 
relative  improvements  could  be  illustrated,  nothing  could  be  said  about  the 
impact  on  the  accuracy  of  location  estimation. 

The  current  study,  expanding  on  the  work  described  at  Reference 
(I),  made  use  of  SAR  incident  data  in  which  the  ELT/EPIRB  had  actually  been 
found.  Using  such  data,  two  areas  of  study  could  be  addressed.  Firstly, 
categorization  techniques  could  be  assessed  based  on  the  type  of  data 
currently  being  handled  by  the  SARSAT  users.  Secondly,  the  impact  of  this 
categorization  in  terms  of  its  meaning  with  regard  to  accuracy  could  be 
quantified. 


2.3  STUDY  OBJECTIVES 


SARSAT  operational  data  categorization  and  accuracy  studies  were 
therefore  initiated  to: 
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.  Validate  categorization  schemes  with  the  aid  of  operational 
SAR  incident  data; 

.  Relate  developed  methodologies  to  error  of  location 
estimation; 

.  Outline  and  recommend  approaches  for  the  actioning  of  SARSAT 
alert  data. 

The  intention  of  the  study  was  to  develop  and  validate  approaches 
which  would  allow  operational  personnel  to  recognize  good,  mediocre  and 
poor  estimation  performance  on  the  part  of  SARSAT  facilities,  and  in 
quantitative  terms  define  the  meaning  of  such  a  categorization.  Given  this 
information,  SARSAT  users  could  then  better  define  their  approach  to  the 
handling  of  the  alert  data. 


3.0  OUTLINE  OF  APPROACH  AND  ANALYTICAL  DEVELOPMENT 


The  approach  adopted  to  expand  on  the  initial  characterization  of 
the  SARSAT  alert  data  was  to  develop  a  data  base  consisting  of  all  those 
SAR  incidents  in  which  SARSAT  played  a  role  and  in  which  the  beacon  was 
located.  This  was  a  different  approach  to  that  used  in  previous  studies. 
In  the  latter  case  a  LUT-to-CMCC  emulator  was  developed,  see  Reference  (4), 
to  simulate  the  flow  of  data  from  the  ground  station  to  the  CMCC  over  a 
period  of  time.  All  data  flowing  through  the  system  was  processed.  For 
the  current  studies,  only  specific  alerts,  i.e.  the  known  SAR  incidents, 
were  run  through  the  simulator.  In  other  words,  each  known  incident  was 
treated  as  the  focus  of  the  emulation,  to  the  exclusion  of  all  other  alert 
data,  and  then  tracked  in  time.  Considerable  effort  was  required  to  build 
this  data  base  as  will  be  described. 

During  the  development  of  the  data  base,  it  was  found  that  the 
data  categorization  schemes  developed  during  the  initial  studies  required 
expansion  to  cover  some  of  the  anomalous  signal  characteristics  being 
observed.  A  review  of  the  current  scheme  is  provided  along  with  the 
modifications  made  to  accommodate  new  signal  types. 

A  significant  aspect  of  the  current  studies  over  that  of  the 
previous  work  is  the  fact  that  the  location  of  the  transmi tting  source  was 
known.  Therefore,  studies  of  system  accuracy  under  operational  conditions 
could  be  carried  out.  However,  as  will  be  discussed,  there  seems  to  be  no 
consistent  approach  within  the  SARSAT  community  for  the  definitions  and 
presentations  of  such  work.  The  rationale  behind  the  approach  adopted  in 
the  current  studies  is  described. 


•.v 
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Therefore,  as  background  to  the  presentation  of  the  results  of 
the  categorization  and  accuracy  studies,  the  data  base  developed  for  these 
studies  is  described,  the  characterization  scheme  definitions  are  updated, 
and  finally,  the  approaches  taken  to  quantify  system  accuracy  are 
di scussed. 


3.1  DATA  BASE  DEVELOPMENT 


During  the  period  of  the  SARSAT  D&E,  the  SARSAT  project  had 
access  to  the  Canadian  SAR  statistical  data  base.  These  data  consist  of 
the  RCC's  records  of  the  information  related  to  all  logged  search  and 
rescue  incidents.  In  order  to  access  this  computerized  data  base, 
retrieval  programs  were  developed  to  identify  and  report  those  cases  of 
particular  interest  to  SARSAT,  specifically,  distress  and  non-distress  SAR 
cases  involving  ELT/EPIRB  transmissions  in  which  SARSAT  facilities  played  a 
role  either  as  the  alerting  device  or  in  a  support  role  by  confirming 
beacon  activation.  The  results  of  these  queries  were  passed  to  the  project 
on  a  monthly  basis  for  analysis.  These  data  were  used  to  support  a  number 
of  studies,  see  Reference  (5),  but  also,  they  provided  a  convenient 
mechanism  to  identify  and  log  those  SAR  incidents  involving  SARSAT.  The 
nature  of  the  incident,  incident  time  and  location  (if  known),  etc.  were 
provided  in  these  reports. 

During  the  period  from  February  1983  to  June  1984,  409  SAR 

incidents  met  the  requirements  noted  above,  i.e.  SARSAT  involvement  and 
knowledge  of  the  transmission  site.  On  average,  this  equates  to  24  events 
per  month.  Table  1  summarizes  these  data  according  to  type. 


TABLE  1 

ELT/EPIRB  INCIDENTS  INVOLVING  SARSAT  DATA 
FEBRUARY  1983  -  JUNE  1984 


ENVIRONMENT 

SARSAT  INVOLVEMENT 
INITIATED  ASSISTED 

SAR  RESOURCES  USED 
YES  NO 

AIR 

DISTRESS 

10 

7 

17 

0 

NON-DISTRESS 

244 

131 

217 

158 

MARINE 

DISTRESS 

1 

1 

2 

0 

NON-DISTRESS 

13 

2 

10 

5 

As  is  evident  from  the  data  in  Table  1,  about  95%  of  the 
incidents  were  non-distress  false  alerts,  i.e.  ELT/EPIRB  activations  for  a 
variety  of  reasons  none  of  which  included  a  distress  or  emergency 
situation.  The  resource  column  indicates  the  number  of  incidents  in  which 
the  SAR  community  expended  time  and  effort  in  the  form  of  sending  a  craft 
(airplane,  helicopter,  rescue  vessel)  out  to  look  for  the  transmitting 
beacon.  Clearly,  on  more  than  one  half  of  the  incidents,  search  and  rescue 
resources  were  used.  Figure  2  illustrates  the  geographic  distribution  of 
these  incidents. 

As  a  routine  activity  during  the  SARSAT  D&E,  the  project 
collected  the  LUT  generated  alert  data  and  stored  them  on  the  SARSAT 

Evaluation  Facility  (SEF)  data  base.  This  data  base  was  a  time  ordered 
structure  containing  SARSAT  evaluation  data  from  a  variety  of  sources.  The 
data  of  interest  for  the  current  studies  were  the  alert  data  consisting  of 
the  LUT  derived  signal  processing  parameters  and  the  estimates  of  location 
resulting  from  the  Doppler  fitting  process  for  the  409  incidents  noted 

above. 

The  SAR  data  and  the  SARSAT  data  were  two  distinct  data  sets 
which  had  to  be  integrated  in  the  following  manner.  The  SAR  incident 
locations  were  used  to  query  the  SEF  data  base  for  the  related  LUT  data, 
using  a  "inhouse"  developed  software  package,  LOCAT,  see  Reference  (6)  for 
additional  detail.  The  basis  of  the  query  was  time  and  location.  The  data 
obtained  by  LOCAT  were  then  processed  through  the  LUT-CMCC  emulator,  see 
Reference  (1)  and  (4).  The  purpose  of  this  process  was  to  "purify"  the 
data  within  a  given  satellite  pass,  that  is,  sidebands  were  combined  by 
means  of  a  distance  and  frequency  bias  criteria  to  produce  one  estimate  of 
location  within  the  pass  for  the  specific  beacon  transmission  of  interest. 
All  other  transmissions  detected  during  the  particular  pass  were  ignored. 

The  next  step  involved  combining  the  data  on  a  pass-to-pass  basis 

emulating  updating  information  coming  into  the  CMCC.  In  order  to  do  this, 

a  number  of  changes  had  to  be  made  to  the  original  LUT-CMCC  emulation 
software.  The  original  software  had  been  written  to  emulate  the  CMCC 
receiving  LUT  data  on  a  continuous  basis  over  a  period  of  time.  ELT/EPIRB 
transmissions  were  detected,  subsequently  confirmed  on  following  satellite 
passes  and  eventually  "aged"  out  of  the  system  when  no  further  detections 
occurred.  For  the  current  studies,  attention  was  focussed  on  a  particular 
transmission  to  the  exclusion  of  all  other  transmissions  within  the  pass. 
Each  SAR  event  of  interest  was  a  separate  query  and  emulation  process. 

Considerable  effort  was  required  to  develop  the  integrity  of  the 
data  base  thus  produced.  Instances  did  arise  wherein  the  data  from  one 
beacon  merged  with  that  from  another  beacon  because  the  initial  query  time 
range  was  too  wide.  This  was  particul ari ly  true  for  ELT/EPIRB 
transmissions  in  or  near  metropolitan  areas.  Eventually,  the  only  approach 
feasible  to  resolve  these  problems  was  to  review  the  anomalous  incidents  on 
a  case-by-case  basis  and  manually  edit  the  data  using  information  available 
from  the  SAR  incident  reports. 


Geographic  Distribution 

SARSAT  Incidents,  Feb  83  -  Jun  84 
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LUT  data  for  380  of  the  409  incidents  noted  in  the  SAR  reports  to  ! 

involve  the  participation  of  SARSAT  were  found  in  the  SEF  data  base.  The  J 

reasons  why  data  for  the  remaining  29  incidents  were  not  found  were  not  i 

established.  However,  a  number  of  plausible  reasons  can  be  put  forward.  j 

In  all  likelihood,  the  most  probable  reason  why  correlating  SARSAT  data 
could  not  be  found  would  be  that  the  originating  SARSAT  source  was  the  USA. 

These  data  would  come  to  Canada  via  the  CMCC  for  onward  transmission  to  the 
RCC's.  The  only  data  interrogated  at  the  SEF  was  the  Canadian  LUT  data. 

The  USA  data  only  provides  location  information,  signal  characterization 
data  is  not  available.  Hence  it  is  of  little  use  to  the  current  studies. 

Another  reason  why  events  may  have  been  missed  is  the  fact  that  the  data 
could  have  been  so  poor  that  it  failed  to  meet  the  initial  query  criteria. 

Since  the  query  was  based  on  a  distance  criteria,  there  was  a  tradeoff  in 
terms  of  making  the  criteria  too  broad  and  hence  contaminating  the  data 
with  other  detections  versus  making  the  criteria  too  narrow  and  therefore  . 

missing  detections.  | 

The  data  base  established  for  the  categorization  and  accuracy 
studies  therefore  consisted  of  the  SARSAT  LUT  data  for  380  SAR  incidents 
over  the  period  of  February  1983  to  June  1984.  A  large  number  of  these 
incidents  involved  detections  on  more  than  one  satellite  pass.  The  total 
number  of  detections  for  these  380  incidents  was  1316  or  roughly  3 
detections  (unique  passes)  per  incident.  These  data  then  were  the  basis 
for  the  analytical  studies  that  are  discussed  in  the  subsequent  1 

presentation. 


3.2  ALERT  DATA  CHARACTERIZATION  SCHEMES  -  A  REVIEW  AND  UPDATE 


The  previous  studies  had  established  the  SARSAT  LUT -CMCC  data 
flow.  In  general,  the  characterization  of  the  SARSAT  alert  data  can  be 
reviewed  as  a  three  step  activity.  Firstly,  all  detected  signals  derived 
by  the  LUT  as  the  result  of  a  satellite  pass  must  be  organized  in  such  a 
manner  that  one  beacon  transmission  results  in  one  detection.  Due  to  the 
electronic  characteristics  of  some  ELTS,  the  LUT  can  derive  a  number  of 
detections  due  to  the  presence  of  strong  signal  sidebands,  or  due  to  the 
absence  of  an  identifiable  carrier.  This  process  of  grouping  the  sidebands 
from  within  a  specific  pass  is  referred  to  as  the  cluster  process.  Once 
this  one-to-one  correspondence  has  been  established  between  the 
transmission  and  the  detection,  a  subset  of  the  available  signal  processing 
parameters  must  be  chosen  as  a  representative  set  for  the  detected 
ELT/EPIRB.  Finally,  this  information  is  transferred  to  the  CMCC  where  it 
is  compared  to  previous  pass  detections  and,  where  appropriate,  updating 
would  occur  using  Kalman  Filter  techniques.  This  latter  process  is 
referred  to  as  the  merge  process. 


Each  of  these  three  steps  are  discussed. 


3.2.1  The  Cluster  Process 


The  LUT  cluster  process  is  a  procedure  to  identify  all  collocated 
transmission  sources  occuring  on  a  given  satellite  pass.  The  purpose  of 
the  cluster  process  is  twofold.  Firstly,  it  is  a  mechanism  to  reduce  the 
volume  of  extraneous  data  which  has  to  be  actioned  by  operational 
personnel.  The  cluster  process  has  been  demonstrated  to  reduce  volume  by 
30%.  Secondly,  the  cluster  size  has  been  found  to  be  quite  a  useful 
parameter  for  categorizing  the  nature  of  the  transmission  source,  given  no 
additional  information. 

The  initial  definition  of  the  signal  type  was  as  given  in 

Table  2, 


TABLE  2 

SIGNAL  TYPE  CATEGORIZATION  (INITIAL) 


SIGNAL  TYPE  CLUSTER  SIZE  (CL) 


U  CL  =  1 

E  1  <CL  $5 

I  CL  >5 


where  U  implied  an  unknown  signal  (single  Doppler  curve),  E  was  taken  to  be 
an  ELT/EPIRB  transmission  with  a  good  carrier  and  a  number  of  sidebands, 
and  I  implied  an  interfering  signal  source  because  of  the  large  number  of 
transmissions  radiating  from  the  same  source. 

During  the  processing  of  the  SAR  incident  data,  a  number  of 
refinements  had  to  be  made  to  the  above  definition  to  further  characterize 
the  I  type  transmissions.  It  was  found  that  multiple  clusters  were  being 
formed  from  a  single  transmission  source. 

Clusters  were  derived  for  the  LUT  alert  data  based  on  a  distance 
and  frequency  bias  check.  In  other  words,  if  a  number  of  transmissions 
were  radiating  from  essentially  the  same  geographic  location  (within  an 
area  of  200  kilometres  radius),  they  were  assumed  to  come  from  a  single 
source.  In  order  to  avoid  the  not  uncommon  situation  of  two  beacons 
functioning  in  close  proximity  to  each  other,  the  estimated  off-centre 
transmission  frequency  or  the  ELT/EPIRB  bias  was  used  as  a  characteristic 
signature  for  the  beacon. 

The  modified  logic  flow  employed  to  classify  the  signal  type  in 
the  current  studies  is  illustrated  in  Figure  3.  The  basic  definition  for 
the  U  and  E  type  signals  remained  the  same  except  that  allowance  was  made 
for  anomalous  asymmetrical  Doppler  plane  patterns  to  be  classified  as  U 
type  signals  as  discussed  below. 


No 


igure  3:  Signal  Characterization  Using 
Cluster  Size  (CL)  and  Bias  (b) 
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Therefore,  using  the  cluster  size  as  the  basic  initial  parameter, 
all  transmissions  wherein  only  one  Doppler  curve  was  identified  were 
classed  as  unknown  or  U  type  signals.  If  the  cluster  size  was  in  the  range 
2  to  5  and  the  frequency  bias  range  was  less  than  4000  Hz,  then  the 
assumption  was  made  that  this  was  characteristic  of  a  good  ELT/EPIRB  and 
hence  the  E  type  signal.  If  during  the  bias  check,  the  criteria  failed  and 
the  cluster  size  was  2,  it  was  felt  that  little  could  be  said  and  the 
signal  was  classified  as  unknown.  This  situation  might  be  indicative  of 
two  poor  transmitting  sources  in  the  same  area. 

With  the  exception  of  the  last  category,  the  above  definitions 
are  basically  the  original  U  and  E  type  classifications.  The  refinements 
made  to  the  I  type  or  interference  type  signal  consisted  of  definitions  for 
N  (narrow  band  noise),  8  (broad  band  noise),  C  (combing  transmissions)  and 
F,  or  faulty  ELT  transmissions. 

The  N  type  transmission  was  defined  to  be  a  cluster  of  size 
greater  than  5  but  with  a  narrow  frequency  bias  range  (4000  Hz).  An  N  type 
transmission  is  really  the  same  as  an  E  type  transmission  but  with  more 
than  the  expected  number  of  sidebands. 

At  this  stage,  all  that  remains  are  the  anomalous  Doppler  plane 
patterns.  The  detections  were  ordered  in  terms  of  bias,  the  first 
difference  was  carried  out  on  this  ordered  list  and  the  bias  range,  R,  the 
mean  bias  5  and  the  mean  bias  difference  calculated,  i.e.  Ab.  If  the 
cluster  size  was  greater  than  10  and  if  the  mean  bias  difference  was 
approximately  equal  to  a  uniform  difference  (~  R/n)  then  the  signal  was 
classified  as  C.  The  test  here  really  was  to  determine  whether  the  Doppler 
curves  were  uniformly  distributed  across  the  observed  frequency  range. 
Failing  this  criteria,  the  signal  was  classified  as  broadband  noise,  or  B. 
The  final  signal  type  was  the  F  type  signal.  Again,  it  was  taken  to  be  a 
Doppler  plane  pattern  consisting  of  5  to  10  curves  with  a  uniform  pattern 
(indigative  of  paired  side  bands).  If  this  criteria  was  not  met,  it  was 
classified  as  unknown. 


During  the  analysis  of  the  SAR  incident  data,  examples  of  the 
above  signal  types,  except  for  the  B  type  signal  which  was  a  default  case, 
were  observed.  Examples  of  the  Doppler  plane  representatives  of  a  C,  F  and 
N  signal  type  are  illustrated  in  Figure  4.  While  the  classification  of 
these  signals  was  that  of  interferer,  it  should  be  noted  that  these 
transmissions  were  from  ELT's  which  were  logged  as  SAR  incidents. 


In  summary,  the  clusters  process,  based  on  cluster  size  and 
frequency  bias,  categories  the  signals  as  one  of  the  types  as  illustrated 
in  Table  3. 
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TABLE  3 

SIGNAL  TYPE  CATEGORIZATION  (UPDATED) 


CRITERIA 

FREQUENCY  BIAS 

SIGNAL  TYPE 

CLASS 

CLUSTER  SIZE 

DISPERSION 

U 

1 

1 

2 

2 

>  4000  Hz 

3 

<  10 

Irregular 

E 

1  <  CL  «  5 

<  4000  Hz 

I 

N 

CL  >  5 

<  4000  Hz 

B 

CL  »  10 

Irregular/Large 

C 

CL  >  10 

Uni  form 

F 

CL  <  10 

Smal 1 ,  Uni  form 

3.2.2  Quality  Parameters 


At  this  stage  of  processing,  the  nature  of  the  transmitting 
signal  has  been  categorized.  Nothing  is  implied  about  the  data  quality. 
Since  the  LUT  Doppler  processing  technique  is  essentially  a  curve 
fitting/template  matching  algorithm,  there  is  associated  with  each  unique 
detection  a  statistical  parameter  set  which  describes  the  goodness-of-fit. 
In  the  case  of  clusters  or  multiple  detections  from  the  same  transmission 
source,  the  criteria  used  to  select  the  representative  parameters  which 
would  best  describe  the  signal  was  that  of  using  the  parameters  associated 
with  the  first  element  in  the  cluster.  The  rationale  for  this  approach  was 
the  fact  that  the  LUT,  during  its  Doppler  processing,  extracts  the  Doppler 
curves  in  order  of  signal  strength.  Therefore,  the  first  element  in  the 
cluster  is  the  strongest  signal  and  hence  can  be  assumed  to  be  the 
carrier. 


The  standard  deviation  of  the  data  residuals,  STD  measured  in 
Hertz,  has  been  observed  from  previous  work  (5)  to  correlate  well  with 
error.  The  following  signal  categorization,  CAT  is  favoured. 


TABLE  4 

SIGNAL  CATEGORY  (CAT)  DEFINITION 


CATEGORY 


DEFINITION  (Hz) 


0  <  STD  <  8 

8  <  STD  «  18 

18  <  STD  «  40 

STD  >  40 


It  is  obvious  that  the  lower  the  STD,  the  better  is  the  match 
between  the  observed  data  and  the  theoretical  Doppler  curve  and  hence,  the 
greater  would  one's  confidence  would  be  in  the  quality  of  the  estimation  of 
ELT  location.  Therefore  a  signal  category  of  A  will  be  better  than  one 
which  is  B,  and  a  D  category  signal  is  very  poor. 

As  the  definition  implies,  the  STD  or  root  mean  square  of 
residuals  says  nothing  about  the  amount  of  data  used  in  the  estimation 
process.  One  could  observe  a  small  STD  based  on  a  few  data  points  leading 
to  a  poor  estimate.  Therefore,  a  quality  factor  Q  was  introduced  as  a 
measure  of  the  density  of  the  Doppler  curve  taking  into  account  geometry 
effects.  It  is  a  measure  of  the  amount  of  data  in  the  curve  relative  to  an 
ideal  curve.  It  ranges  in  value  from  0  to  1,  one  being  equated  as  perfect 
in  terms  of  data  density  (but  not  necessarily  goodness-of-fit) .  For 
convenience,  Q  is  given  in  terms  of  its  quartiles.  The  definition  of  Q  is 
provided  at  Reference  1  . 

The  argument  is  then  made  that  the  signal  category  and  the  Q 
factor  are  reasonable  guides  for  assessing  the  quality  of  the  SARSAT  alert 
data.  The  objective  of  the  analysis  discussed  in  Section  4  is  the 
development  of  operational  rules  usings  the  signal  type,  CAT  and  Q  to  guide 
user  actioning  of  the  alert  data. 

It  is  well  understood  that  the  ELT/EPIRB-to-satell ite  geometry 
has  an  impact  on  data  quality.  Overhead  satellite  passes  give  an  ambiguity 
in  the  latitude  estimate  while  beacons  transmitting  close  to  the  satellite 
viewing  horizon  present  problems  in  longitude.  For  the  COSPAS  and  SARSAT 
satellites  (altitudes  in  the  range  800-1100  kilometres),  it  has  been 
established  that  the  system  functions  reasonably  well  in  the  cross-track 
angle  (CTA)  range  2°$CTA<18°.  Outside  this  range,  system  performance  can 
degrade  significantly.  Therefore,  the  procedures  developed  for  actioning 
SARSAT  alert  data  must  take  into  account  the  negative  impact  that  poor 
geometry  has  on  the  estimation  process. 
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3.2.3  The  Merge  Process 


Thus  far,  the  discussion  has  focussed  on  categorizing  the  LUT 
alert  data  for  a  given  satellite  pass.  However,  on  a  large  number  of  SAR 
incidents,  the  signal  is  detected  on  subsequent  passes.  This  gives  rise  to 
the  need  to  combine  current  pass  data  with  previous  pass  data.  Since  the 
LUT  Doppler  processing  techniques  used  to  fit  the  observed  data  to  the 
theoretical  model  are  statistical  procedures,  i.e.  least  squares  fitting, 
sufficient  data  is  available  to  support  a  Kalman  Filter  updating  process, 
see  Reference  (1)  for  details. 

In  an  analogus  manner  to  the  formation  of  clusters  at  the  LUT, 
current  pass  detections  at  the  CMCC  can  be  compared  to  previous  pass 
detection  based  on  distance  and  frequencing  bias  criteria.  If  the  match  is 

successful ,  the  data  from  the  passes  can  be  merged  to  form  a  new  "best 

estimate"  of  the  location  of  the  transmitting  source.  In  an  operational 
scenario,  this  process  would  continue  until  such  time  as  the  beacon  was 
located  through  SAR  actioning  and  turned  off  or  the  owner  of  the  ELT/EPIRB, 
for  any  number  of  reasons,  discovered  that  his  ELT/EPIRB  was  on  and  turned 
it  off  himself.  In  the  latter  case,  the  incident  would  be  categorized  as  a 
detected  but  not  located  false  alert. 

The  merge  process  gives  rise  to  two  sets  of  location  estimates. 
The  first  set  is  the  LUT  estimates  which  are  based  solely  on  data  from  an 

individual  satellite  pass.  The  second  is  the  CMCC  data  which  are  based  on 

multiple  pass  detections.  The  latter,  theoretically,  will  provide  better 
location  estimates.  The  quantitative  improvement  of  the  CMCC  data  over 
that  of  the  LUT  data  for  the  380  SAR  incidents  which  form  the  analysis  data 
base  are  discussed  in  Section  4. 


3.3  ACCURACY  MODELS 


There  is  no  consistent  approach  within  the  SARSAT  community  for 
the  definition  of  accuracy  of  location  estimation.  One  discussion,  see 
Reference  (7),  attempted  to  classify  the  definition  of  accuracy,  but  to 
date,  different  evaluation  teams  are  using  different  approaches. 

These  problems  of  definition  occur  for  a  number  of  reasons,  two 
of  which  are  as  follows.  The  primary  reason  is  related  to  the  ambiguity 
arising  from  the  initial  engineering  specification  of  performance  of  the 
LUT.  As  a  design  goal,  documentation  specified  accuracy  as  20  kilometres, 
one  sigma.  This  would  seem  at  first  to  be  a  reasonable  definition  given 
that  one  has  a  univariate,  symmetrical  or  normal  distribution  of  error. 
However,  SARSAT  involves  detections  in  a  region  (error  in  latitude  and 
longitude),  obviously  bivariate.  The  reduction  of  the  bivariate  error 


model  to  a  univariate  model  through  the  use  of  the  radial  error  leads  to  a 
situation  in  which  the  specification  requirement  for  accuracy  does  not  make 
sense.  The  second  reason  why  problems  have  occurred  is  that  evaluators  are 
looking  for  simple  accuracy  qualifying  parameters  which  will  describe  the 
system  in  its  most  favourable  terms.  On  average,  SARSAT  performs  quite 
well.  However,  the  range  of  performance  in  the  operational  environment  is 
very  broad,  due  in  large  measure  to  the  poor  performance  characteristics  of 
the  ELT/EPIRB.  In  other  words,  as  discussed  previously,  it  can  function 
extremely  well,  but  also  it  can  give  quite  poor  performance.  This  latter 
performance  then  tends  to  bias  the  interpretation  of  system  performance 
averages. 

At  Reference  (7),  it  is  argued  that  within  geometric  constraints 
noted  previously,  the  SARSAT  estimation  process  will  demonstrate  normally 
distributed  errors  in  latitude  and  longitude.  Furthermore,  it  is  assumed 
that  the  magnitude  of  error  in  latitude  and  longitude  is  the  same  and  that 
error  is  independent. 

Therefore,  under  these  assumptions  it  follows  that,  letting 


Xg  =  error  in  latitude,  ~N  (^x,  ox) 
Ye  =  error  in  longitude,  ~N  (^y,  cy) 

and  ax  =  ay  =  a,  then  if 


r  =  /  X2e  +  Y2e,  the  radial  error 


it  is  quite  simple  to  illustrate  that  r,  the  radial  error  is  distributed  as 
a  Rayleigh  distribution, 


The  cumulative  probability  function  for  radial  error  less  than 
some  R  then  is 


Pr(r  R)  =  i_e-R2/2a2 


and  the  first  and  second  moments  of  the  distribution,  expressed  in  terms  of 
a  are 


Mr  ■ 

K 

a,  2  =  2  a2  ( 1  -1  /  n) 


A  number  of  problems  exist  in  the  above  development.  Firstly, 
the  assumptions  of  error  equality  and  independence  can  be  questioned.  It 
is  known  that  the  Doppler  curve  fitting  process  is  more  sensitive  in 
latitude  estimation  than  longitude  estimation.  Therefore,  it  follows  that 
errors  in  latitude  would  be  less  than  those  in  longitude.  Independence  is 
doubted  in  the  operational  environment.  Work  on  simulation  studies  have 
suggested  that  the  more  the  system  departs  from  the  ideal  Doppler  curve 
fitting  situation,  the  weaker  the  assumption  of  error  independence  becomes. 
As  is  illustrated  in  Figure  4,  the  operational  environment  with  low  power 
ELT/EPIRB  and  questionable  transmission  characteristics  is  far  from  ideal. 

Secondly,  referring  back  to  the  specification  for  accuracy,  there 
is  an  ambiguity  over  which  sigma  to  use,  the  a  from  the  bivariate  normal  or 
the  ar  for  the  radial  error. 

If  R  =  a,  then  Pr  ( r  <  R  )  =  0.39,  but  if  R  =  o_  then 
Pr  (r<  R)  =  0.49.  This  is  obvious  that  the  use  of  a  in  the  specification 
of  accuracy  is  liberal,  only  40$  of  the  detections  need  be  within  one 
sigma.  On  the  other  hand,  the  use  of  a-  leads  to  a  general  approach  in 
which  the  median  of  the  cumulative  distribution  can  be  used. 

In  order  to  circumvent  the  problems  noted  above,  the  approach 
adopted  for  the  analysis  discussed  in  Section  4  was  a  distribution  free 
approach.  This  approach  requires  no  assumptions.  Instead,  observed 
probability  distribution  functions  were  used.  Since  the  preferred  measure 
of  error  is  the  radial  error  which  in  turn  gives  rise  to  an  asymmetrical 
density  function,  the  distribution  characteristic  was  taken  to  be  the 
median  or  50  percentile.  It  is  an  easily  quantifiable  parameter  and 
coincidentally  closely  corresponds  to  Pr  (r  <  R)  when  R  =  ar.  As  an  added 
descriptor,  the  percentile  at  which  error  is  less  than  20  km  is  also 
provided  as  a  relative  measure. 


4.0  ANALYTICAL  RESULTS 


Using  the  approaches  discussed  in  Section  3,  an  extensive 

analysis  was  carried  out  on  the  1316  detections  resulting  from  the  380  SAR 

incidents.  Firstly,  the  data  was  characterized  in  terms  of  the  signal 

distribution  and  the  quality  parameters.  Observed  location  error  was 
studied  in  terms  of  this  characterization.  Then,  a  data  categorization 
scheme  was  proposed  and  evaluated.  Finally,  the  impact  of  the  Kalman 

Filter  on  the  parameter  and  location  estimate  were  considered  with  regard 
to  this  categorization  scheme. 
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4.1  INCIDENT  DATA  CHARACTERIZATION 


The  primary  data  characterizing  parameters  are  the  signal  type, 
the  signal  category  parameter  and  the  Q  factor.  The  distribution  data  for 
these  parameters  are  discussed. 


4.1.1  Signal  Type  Distribution 


It  has  been  argued  that,  in  the  absence  of  coll aborating 
information,  the  signal  type  is  a  reasonable  indicator  of  the  nature  of  the 
radiating  source.  Figure  5  illustrates  the  signal  type  distribution  among 
the  three  major  categories,  U,  E,  and  I  for  the  1316  detections.  Clearly, 
more  than  60%  of  the  detections  are  classified  as  U  type  signals  while  less 
than  5%  are  classified  as  I.  Because  of  the  nature  of  the  I  type  signal, 
and  its  relative  infrequency,  it  is  grouped  together  with  the  E  type  data 
for  statistical  purposes. 

Therefore,  presumably  due  to  the  wide  variation  in  the 
performance  of  the  existing  generation  of  ELT/EPIRBs,  only  35%  of  the 
signals  fall  within  the  norm  expected  for  those  radiating  as  an  ELT/EPIRB. 

Now,  the  critical  issue  for  the  380  SAR  incidents  is  that  of 
establishing  what  information  can  be  derived  from  the  first  detection  and 
then,  what  confirmation  information  is  provided  by  the  second  detection. 
The  signal  characteristics  for  subsequent  detections  become  progressively 
less  important  since  confirmation  data  is  now  available  from  multiple 
passes. 


As  will  be  discussed  in  Section  4.4,  291  of  the  380  SAR  incidents 
involved  multiple  pass  detections  or  SARSAT  confirmations.  Table  5 
summarizes  these  291  cases  in  terms  of  signal  type  for  the  first  detection 
and  then  the  subsequent  signal  type  on  the  second  detection. 


TABLE  5 

SIGNAL  TYPE  -  MULTIPLE  DETECTIONS 


SIGNAL  TYPE 

FIRST  DETECTION 

SIGNAL  TYPE 

SECOND  DETECTION 

U 

165 

U 

98 

E  or  I 

67 

E  or  I 

126 

U 

71 

E  or  I 

55 

It  is  interesting  to  note  that  the  proportion  of  U  type  signals 


by  SAR  incident  is  the  same  as  that  noted  for  detection,  i.e.  about  60%. 
Furthermore,  this  ratio  of  6  to  4  carries  through  to  the  second  detection. 
It  is  also  evident  chat,  at  most,  the  signal  type  is  a  weak  ELT 
characteristic.  The  signal  type  classification  can  change  from  one  pass  to 
the  next.  Using  basic  Bayes  statistics,  it  can  be  inferred  from  Table  5 
that  the  probability  that  a  U  type  signal,  noted  on  the  first  pass,  will  be 
classified  as  a  U  type  signal  on  the  next  pass  is  0.34.  Furthermore,  there 

is  a  50-50  chance  that  the  signal  type  categorization  will  change  from  one 

pass  to  the  next.  At  this  stage  of  the  discussion,  these  comments  do  not 

take  into  account  the  quality  of  the  data,  e.g.  a  poor  detection  on  the 

first  pass  giving  rise  to  a  U  type  characterization  followed  by  a  good 
detection  on  the  next  pass  giving  rise  to  an  E  or  I  type  characterization. 

Considerable  effort  was  made  to  classify  the  interference  type 
signals  as  described  in  Section  3.  While  they  only  constitute  about  5%  of 
the  detections  (57  of  the  1316  detections),  it  is  felt  that  operationally, 
the  subcategorization  is  useful  because  it  indicates  a  problem  or  abnormal 
type  signal  source.  Generally,  it  can  be  immediately  concluded  that  any  I 
type  signal  is  a  true  signal  source  with  identifiable  characteristics  as 
noted  by  the  subcategorization.  Figure  6  illustrates  the  observed 
distribution  of  the  I  type  signals.  About  50%  of  these  signals  were  F 
type,  just  under  40%  were  N,  and  the  balance  were  C. 
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Figure  6:  Interferers  Distribution. 


In  summary,  it  is  concluded  that  the  signal  type  indicator 
provides  a  reasonable  image  of  the  signal  source  and  does  suggest,  in  a 
straightforward  manner,  the  nature  of  the  Doppler  data  used  to  derive  the 
location  estimate.  In  general,  about  60%  of  the  detections  will  be 
classified  as  U  type  signals,  35%  as  E  type  and  the  remaining  51  as 
interferers.  At  most,  the  signal  type  is  a  weak  characterizing  parameter 
on  a  pass-to-pass  basis.  However,  this  weakness  becomes  less  significant 
as  confirmation  of  detection  becomes  available.  The  knowledge  of  the 
signal  type  will  help  indicate  to  the  SAR  forces  the  problems  they  might 
encounter  on  their  final  homing  on  the  ELT/EPIRB. 


4.1.2  Signal  Category  Distribution 


The  next  parameter  of  interest  is  the  signal  category.  It  is 

directly  related  to  the  standard  deviation  of  the  Doppler  curve  and 
therefore  is  a  measure  of  goodness-of-fit  of  the  signal  processing 

algorithm. 

Figure  7  illustrates  the  distribution  of  signal  category  for  the 
1316  detections  in  the  data  base.  These  data  suggest  that  about  30%  of  the 
data  is  of  poor  quality,  indicative  of  the  nature  of  the  quality  of  the 
ELT/EPIRBs  SARSAT  is  expected  to  locate.  In  30%  of  the  cases,  the 
transmission  source  is  of  good  quality,  and  the  remaining  40%  fall 
somewhere  in  between.  The  problem  is  to  recognize  the  poor  quality 

transmission  and  then,  in  quantitative  terms,  state  what  this  should  mean 
to  SAR  personnel  who  have  to  action  the  data. 

Figure  8 (a-c )  correlates  signal  category  by  signal  type.  It  is 
obvious  that  the  ordering  of  signal  type  by  category  in  terms  of  poorest  to 
best  estimate  is  U,  E  and  I,  a  fact  noted  in  previous  studies.  Therefore 
it  can  be  concluded  that  the  signal  type  is  a  valid  indicator  of 
goodness-of-fit  and  in  combination  with  the  signal  category  parameter  can 
be  used  to  classify  incoming  detection  data. 


4.1.3  Q  Factor  Distribution 


The  third  data  quality  parameter  of  interest  is  the  Q  factor,  the 
parameter  which  is  indicative  of  the  amount  or  density  of  the  data. 

In  a  similar  approach  as  taken  with  the  signal  category  data,  Q 
was  plotted  for  the  total  data  set,  and  then  plotted  by  signal  type.  These 
plots  are  given  as  Figure  9  and  10(a-c). 


Figure  7:  Signal  Category 
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Figure  10:  Q  Factor  by  Signal  Type 


Once  again,  the  poor  quality  of  the  transmitting  sources  is 
evident.  Figure  9  indicates  that  in  almost  50%  of  the  detections,  less 
than  one  half  of  the  expected  data  is  available.  The  LUT  is  working  with 
75%  or  more  of  the  expected  data  in  only  20%  of  the  detections. 

Figure  10{a-c)  suggests  the  problems  associated  with  identifying 
the  type  of  signal.  With  less  data,  the  poorer  the  signal  characterization 
becomes.  It  is  obvious  that  the  characterization  of  the  signal  type  is 
highly  dependent  on  the  quality  of  the  incoming  signal.  This  implies  that 
the  stronger  the  signal,  the  more  is  the  likelihood  that  sidebands  will  be 
above  the  noise  threshold  and  hence  the  more  likely  that  the  transmission 
would  be  categorized  as  something  other  than  U. 

Thus,  based  on  the  above  argument,  there  is  support  to  suggest 
that  the  type  categorization  is  a  good  transmitting  source  characteristic, 
but  that  it  will  only  be  a  consistent  characteristic  under  good  detection 
conditions.  Thus  for  the  current  generation  of  ELT/EPIRBs  which  are  very 
close  to  the  SARSAT  detection  threshold,  inconsistencies  will  be  observed 
from  pass-to-pass  as  noted  in  Section  4.1.1.  However,  it  is  equally 
evident  that  once  a  signal  is  identified  as  a  E  or  I  type  transmission,  it 
is  a  good  signal  and  can  be  actioned  with  some  confidence. 


4.2  ACCURACY  OF  ESTIMATION 


It  would  seem  intuitively  evident  that  the  location  estimate  of 
the  ELT/EPIRB  would  be  better  for  E  and  I  type  transmission  than  for  U  type 
signals.  Similarily,  the  smaller  the  standard  deviation  and  the  larger  the 
0  factor,  the  smaller  one  would  expect  the  error  in  location  estimation. 
Furthermore,  the  SARSAT  system  is  not  expected  to  estimate  the  location  of 
the  ELT/EPIRB  with  uniform  precision  for  all  cross-track  angles. 

Figure  ll(a-d)  contain  the  cumulative  error  distribution 
functions  for  each  of  parameters  discussed  above.  These  diagrams 
illustrate  the  parameter  definition  consistency  in  terms  of  location 
accuracy.  The  E  and  I  type  transmissions  are  consistently  better  than  the 
U  type  transmissions.  However,  some  30%  of  the  U  type  transmissions  do 
result  in  good  location  estimation.  Similarily,  40%  of  the  E  and  I  type 
transmissions  can  result  in  poor  estimation. 

When  the  standard  deviation  is  low,  e.g.  the  A  category,  system 
accuracy  is  very  good.  As  noted  in  Figure  10(b),  error  increases  rapidly 
as  the  category  goes  from  B  to  C/0.  In  a  like  manner,  the  Q  factor 
categories  correlate  very  well  with  error.  In  other  words,  the  greater  the 
amount  of  data,  the  better  the  estimation  of  location.  Finally, 

Figure  11(d)  illustrates  the  impact  of  CTA  on  location  estimation  and  tends 
to  validate  the  choice  of  CTA  "break-points". 
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These  distribution  functions  are  characteristic  of  Rayleigh  type 
distributions  as  would  be  expected  when  using  the  radial  error.  There  is  a 
high  concentration  of  quality  data  at  the  lower  end  of  the  distribution 
with  a  large  spread  of  poor  quality  over  the  long  tail  of  the 
distribution. 

These  distributions  are  characterized  in  Table  6  in  terms  of  the 
50  percentile  of  the  distribution  and  in  terms  of  the  percentile  wherein 
error  of  location  estimation  is  less  than  20  kilometres. 

It  is  obvious  from  the  data  given  in  Table  6  and  illustrated  in 
Figure  ll(a-d)  that  system  performance  cannot  be  characterized  in  an  simple 
manner.  For  example,  if  operational  personnel  were  to  base  actioning 
solely  on  signal  type,  i.e.  action  E  and  I  type  transmissions,  35%  of  the  U 
type  transmissions  from  the  SAR  incidents  have  observed  error  of  less  than 
20  kilometres.  Similarily,  while  the  category  flag  is  the  best  indicator 
of  data  quality,  good  detections,  albeit  a  small  percentage,  can  be  made 
when  the  category  flag  indicates  poor  quality  data.  Both  the  Q  factor  and 
the  geometry  flags  are  good  data  qualifiers,  but  as  with  the  above,  they 
cannot  be  used  in  isolation. 

Therefore,  it  is  concluded  that  the  data  quality  indicators 
described  in  Section  3  are  all  valid  and  consistent  measures  of  the 
performance  of  the  SARSAT  system,  and  they  are  good  indicators  of  how  SAR 
personnel  should  interpret  and  action  SARSAT  alert  data.  It  is  further 
concluded  that  in  order  to  interpret  the  quality  of  incoming  alert  data, 
these  parameters  should  be  used  in  combination.  Such  a  combination  is  the 
subject  of  the  discussion  in  the  next  section. 


TABLE  6 

ERROR  DISTRIBUTION  PARAMETERS  VERSUS  SIGNAL  CHARACTERIZATION 


SIGNAL 

NO.  IN 

DISTRIBUTION 

PERCENTILE  ERROR 

CHARACTERISTIC 

CLASS. 

50  PERCENTILE  (KMS ) 

LESS  THAN  20  KM 

TYPE 

E  OR  I 

507 

24.9 

45.8 

U 

809 

64.0 

35.3 

CATEGORY 

A 

417 

10.8 

60.3 

B 

512 

32.7 

42.2 

C/0 

387 

102.8 

12.9 

Q  FACTOR 

4 

266 

15.6 

53.0 

3 

418 

23.8 

47.0 

2 

242 

56.3 

34.4 

1 

390 

111.7 

19.0 

CTA  FLAG 

-1 

129 

77.5 

26.7 

0 

1027 

28.7 

44.6 

1 

160 

101.7 

19.7 

4.3  ALERT  DATA  CATEGORIZATION  SCHEME 


At  Reference  (1),  a  preliminary  alert  data  categorization  scheme 
was  proposed.  It  basically  consisted  of  defining  "good"  data  as  that  which 
was  classified  as  being  category  A  or  B  with  a  Q  factor  greater  than  0.5. 
"Bad"  data  consisted  of  those  alerts  observed  to  have  standard  deviations 
in  the  C  or  D  class  and  a  Q  factor  less  than  or  equal  to  0.5.  "Mediocre" 
data  then  was  defined  to  be  not  "good"  or  not  "bad".  The  criteria  for 
validating  this  approach  was  whether  the  scheme  could  isolate  the  alert 
data  by  signal  type.  This  is  a  reasonable  approach,  but  as  noted  in  the 
previous  section,  errors  in  classification  can  be  made. 

The  current  studies  have  the  benefit  of  the  location  data 
associated  with  the  detection.  Therefore,  the  categorization  scheme  can  be 
developed  and  quantified  in  terms  of  location  accuracy. 

Using  the  SAR  incident  data,  the  original  scheme  was  modified  and 
the  following  defintions  developed: 

Good  Estimate:  CTA  =  -1,1  and  Category  A  and  Q  =  4 

or  CTA  =  0  and  Category  A  or  B  and  Q  =  3  or  4. 

Mediocre  Estimate:  CTA  =  -1,1  and  Category  A  and  Q  =  3 

or  CTA  =  0  and  Category  A  or  B  and  Q  =  1,2 

Poor  Estimate:  Not  good  and  not  mediocre 


The  data  in  Table  6  and  that  illustrated  in  Figure  ll(a-d) 
provide  a  means  of  ranking  the  quality  indicators,  and  this  is  reflected  in 
the  above  definitions.  Good  estimates  can  be  derived  under  fringe  geometry 
conditions  if  the  signal  is  strong.  If  the  pass  geometry  is  good,  then  one 
still  should  expect  good  location  accuracy  for  slightly  less  stringent 
goodness-of-fit  and  data  density  conditions.  Hence  the  definition  of  the 
of  the  good  estimate.  The  definition  of  mediocre  estimates  reflects  the 
ranking  of  the  quality  indicators.  The  category  flag  reflects  good 
estimation,  but  the  Q  factor  is  down.  One  is  now  in  the  nebulous  position 
of  having  a  good  curve  match  but  less  data.  A  poor  estimate  is  then 
defined  as  data  neither  classified  good  or  mediocre.  It  tends  to  emphasize 
the  extremes  of  the  scales. 

While  the  alert  data  categorization  scheme  given  above  appears 
intuitatively  logical,  it  took  considerable  effort  to  balance  the  break 
points  between  the  three  quality  parameters. 

The  validity  of  the  categorization  is  dependent  on  two  factors. 
Firstly,  does  the  categorization  partition  the  data  base  into  an  acceptable 
balance?  It  is  futile  to  categorize  the  data  as  all  being  good  or  all 
being  bad.  Secondly,  what  does  this  categorization  mean  in  terms  of 
accuracy? 


Table  7  quantifies  the  partition  of  the  1316  detections  arising 
from  the  380  SAR  incidents  by  signal  type  according  to  the  alert 
categorization  scheme.  The  numbers  in  brackets  are  the  percent  breakdowns 
of  the  totals  by  data  category. 


TABLE  7 

ALERT  DATA  CATEGORIZATION  BY  SIGNAL  TYPE 


DATA  CATEGORY 

U  TYPE 

E  TYPE 

I  TYPE 

TOTAL 

GOOD 

245  (30%) 

262  (58%) 

41  (72%) 

548  (42%) 

MEDIOCRE 

185  (23%) 

84  (19%) 

4  (  7%) 

273  (21%) 

POOR 

379  (47%) 

104  (23%) 

12  (21%) 

495  (37%) 

809  (62%) 

450  (34%) 

57  (  4%) 

1316 

The  data  in  Table  7  suggests  that  the  partitioning  of  the  data  by 
the  alert  categorization  scheme  is  reasonable  and  is  in  reasonable 
proportions.  It  only  remains  to  demonstrate  that  the  error  distribution 
functions  reflect  and  validate  the  scheme,  and  that  poor  data  is  observed 
(in  terms  of  error)  to  be  poor  while  good  data  is  in  fact  good. 

Figure  12  illustrates  the  error  distribution  functions  for  the 
LUT  data  according  to  the  alert  data  categorization  scheme.  Table  8 
contains  the  distribution  parameters  for  the  three  categories. 


TABLE  8 

ERROR  DISTRIBUTION  PARAMETERS  BY  DATA  CATEGORIZATION  SCHEME 


NO.  IN 

DISTRIBUTION 

PERCENTILE  ERROR 

DATA  CATEOGRY 

CLASS 

50  PERCENTILE  (KMS ) 

LESS  THAN  20  KM 

GOOD 

548 

13.9 

55.6 

MEDIOCRE 

273 

20.3 

49.8 

BE 


LUT  ESTIMATION  CRITERIA 


/ 


/ 

/ 

/ 

/ 

/ 


4  6  8  10  12  14  16 

Error  (x  10  Kilometres) 


Figure  12:  Error  Distribution  by 
Data  Categorization. 


It  is  apparent  that  the  alert  data  categorization  scheme  does  in 
fact  isolate  out  the  data  according  to  quality  in  a  consistent  manner.  The 
one  remaining  area  to  investigate  is  the  impact  of  the  Kalman  Filter  on  the 
pass-to-pass  merge  and  updating  process.  This  is  discussed  in  the  next 
section. 


4.4  IMPACT  OF  THE  KALMAN  FILTER  -  THE  CMCC  DATA 


Thus  far,  the  discussion  has  concentrated  on  analyzing  the  impact 
of  alert  data  characterization  schemes  on  the  SARSAT  LUT  data.  In  an 
operational  scenario,  SARSAT  facilities,  in  many  cases,  will  detect  the 
transmitting  ELT/EPIRB  over  a  number  of  satellite  passes.  This  gives  rise 
to  multiple  estimates  of  location  for  the  same  beacon  and  hence  the  need 
for  Kalman  Filter  techniques  to  combine  these  estimates  to  produce  a 
current  best  estimate  of  location  at  any  given  point  in  time. 

The  SAR  incident  data  base  consisted  of  380  events  of  which  89 
were  found  to  be  single  detection  incidents  (based  on  the  Canadian  LUT  data 
only).  This  leaves  291  incidents  involving  multiple  detections.  The 
distribution  of  the  number  of  merges  is  illustrated  in  Figure  13.  As  noted 
above,  about  one  quarter  of  the  incidents  were  single  detections  and 
approximately  25%  involved  only  one  additional  detection.  Therefore  50%  of 
the  incidents  involved  two  or  more  detections. 

The  single  detection  incidents  consisted  of  51  U  type  signals, 
34  E  type  signals  and  4  I  type  signals.  In  proportional  terms,  60%  were  U 
and  about  40%  were  E  or  I,  the  common  partition  noted  previously.  Since 
these  data  contribute  nothing  to  the  understanding  of  the  impact  of  the 
Kalman  Filter,  they  were  ignored  and  the  analytical  effort  was  concentrated 
on  the  remaining  219  incidents  involving  1,227  detections. 

Figure  14  illustrates  the  error  distributions  for  the  LUT  and  the 
CMCC  (or  filtered  data)  not  taking  into  account  the  conditions  of  the 
detections.  Not  surprisingly,  the  CMCC  data  provides  a  better  overall 
estimate  of  the  location  of  the  ELT/EPIRB.  Furthermore,  as  would  be 
expected,  the  filter  does  little  to  improve  good  data  and  is  incapable  of 
correcting  poor  data.  Its  most  significant  impact  is  in  the  region  when 
error  of  estimation  is  in  the  range  20-100  kilometres.  In  this  range,  the 
Kalman  Filter  can  reduce  error  in  the  order  of  12%,  again  not  taking  into 
account  the  conditions  of  the  detection.  Table  9  contains  the  parameters 
of  the  two  distributions. 


TOTAL  DATA  SET 


TABLE  9 


ERROR  DISTRIBUTION  PARAMETERS  FOR  CMCC  AND  LUT  DATA 


DATA 

DISTRIBUTION 


DISTRIBUTION 
50  PERCENTILE  (KMS ) 


PERCENTILE  ERROR 
LESS  THAN  20  KM 


CMCC 

LUT 


30.9 

41.8 


41.7 

39.3 


At  the  distribution  50  Percentile,  accuracy  of  location  can  be 
improved  by  about  10  kilometres.  However,  as  noted  above,  for  good 
location  estimates,  e.g.  error  less  than  20  kilometers,  the  improvement  is 
smal 1 . 


Figure  11  and  Table  6,  see  Section  4.2,  contained  the  error 
distribution  parameters  categorized  by  signal  characteristics,  i.e  signal 
type,  category,  Q  factor  and  CTA.  Table  10  contains  the  same  data  as  well 
as  the  data  calculated  for  the  CMCC  distributions. 


TABLE  10 


ERROR  DISTRIBUTION  PARAMETERS 
VERSUS 

SIGNAL  CHARACTERIZATION  (CMCC  &  LUT) 


SIGNAL 

CHARACTERISTIC 

TYPE  E  OR  I 

U 

CATEGORY  A 

8 

C/D 

Q  FACTOR  4 

3 
2 
1 

CTA  FLAG  -1 


DISTRIBUTION 
50  PERCENTILE  (KMS) 


CMCC 

LUT 

A 

24.9 

24.9 

0.0 

35.9 

64.0 

-28.1 

13.2 

10.8 

2.4 

33.5 

32.7 

0.8 

52.0 

102.8 

-50.8 

19.2 

15.6 

3.6 

24.3 

23.8 

0.5 

35.6 

56.3 

-20.7 

54.3 

111.7 

-57.4 

51.2 

77.5 

-26.3 

PERCENTILE  ERROR 

LESS 

THAN  20 

KMS 

CMCC 

LUT 

A 

45.2 

45.8 

-0.6 

39.5 

35.3 

4.2 

57.1 

60.3 

-3.2 

39.8 

42.2 

-2.4 

27.5 

12.9 

14.6 

50.6 

53.0 

-2.4 

45.8 

47.0 

-1.2 

38.8 

34.4 

4.4 

29.3 

19.0 

10.3 

27.9 

26.7 

1.2 

These  data  further  emphasize  that  which  was  stated  previously. 
The  filter,  as  expected,  does  little  to  improve  quality  data  but  has  a 
significant  impact  on  the  poorer  quality  detections.  In  some  of  the  signal 
characteristic  categories,  the  quantitative  improvement  is  quite  major. 
Figure  15(a-d)  contains  the  error  distribution  plots  of  the  filter  data  for 
each  of  the  data  quality  parameters. 

The  one  remaining  item  to  illustrate  is  the  impact  of  the 
filtered  data  on  the  alert  data  characterization  scheme.  Table  11 
summarizes  the  error  distribution  parameters  for  the  CMCC  and  the  LUT  data 
for  the  categorization  scheme  defined  in  Section  4.3. 


TABLE  11 


ERROR  DISTRIBUTION  PARAMETERS  BY  DATA  CATEGORIZATION  SCHEME 

FOR  CMCC  AND  LUT  DATA 


NO.  IN 

DISTRIBUTION 

PERCENTILE  ERROR 

DATA  CATEGORY 

CLASS 

50  PERCENTILE 

(KMS) 

LESS  THAN  20 

KMS 

CMCC 

LUT 

A 

CMCC  LUT 

A 

GOOD 

548 

18.0 

13.9 

4.1 

51.7  55.6 

-3.9 

MEDIOCRE 

273 

25.8 

20.3 

5.5 

45.2  49.8 

-4.6 

POOR 

495 

50.7 

96.9  - 

■46.2 

28.6  15.6 

13 

Again, 

the  impact 

of  the 

Kalman 

Filter 

is  evident. 

It  has 

little,  or  to  some  extent,  a  negative  impact  of  good  and  mediocre 
classified  data  but  a  major  impact  on  the  poorer  quality  data.  At  the 
distribution  50  Percentile,  for  the  poor  quality  data,  one  is  looking  at 
improvements  in  the  order  of  50  kilometres.  Furthermore,  the  number  of 
poor  quality  detections  with  observed  errors  of  less  than  20  kilometres  has 
doubled. 

Figure  16  contains  the  error  distribution  functions  for  these 
data  in  terms  of  the  alert  categorization  scheme  definitions.  If  one 
compares  these  data  to  that  given  in  Figure  11,  it  is  readily  apparent  that 
the  data  distributions  for  the  good  and  mediocre  data  is  essentially  the 
same  (the  impact  of  the  filter  is  to  smooth  the  distribution  curve)  while 
that  for  the  poor  category  is  altered  and  improved  significantly.  The 
differences  between  these  two  distributions  is  illustrated  in  Figure  17. 

In  summary,  it  is  evident  that  the  Kalman  Filter  significantly 
improves  the  accuracy  of  estimation  of  the  detections  derived  by  the  SARSAT 
LUT.  This  improvement,  overall,  is  insignificant  if  the  original  estimates 
were  of  good  quality.  However,  major  improvement  is  noted  from  poorer 
quality  estimates,  and  since  about  40%  of  the  SARSAT  data  fall  into  this 
latter  category,  the  importance  of  filtering  the  data  cannot  be 
understated. 


3 


CMCC  ESTIMATION  CRITERIA 


G 

^  //p 
/ 

/ 


* ■  / 


0  2  4  6  8  10  12  14  16  18  20 

Error  (x  10  Kins) 


Figure  16:  Error  Distribution 

CMCC  Data  by  Category 


-  38  - 


5.0  DISCUSSION  OF  RESULTS 


As  a  result  of  the  approaches  outlined  in  Section  4,  it  is 
evident  that  the  SARSAT  facilities  can  better  support  the  user  and  can 
provide  significant  guidance  on  how  best  to  action  the  ELT/EPIRB  alert 
data.  This  guidance  comes  through  a  judicious  use  of  existing  data. 

The  issue  in  Section  4  was  to  be  able  to  identify  or  categorize 
alert  data  in  a  meaningful  manner  so  that  the  SAR  user  has  sufficient 
information  to  make  a  reasonable  and  balanced  decision  on  whether  to 
respond  immediately  to  the  data  or  to  await  further  information.  In 
summary,  definitions  can  be  used  to  distinguish  good,  mediocre  and  poor 
SARSAT  data.  The  question  is  how  best  to  action  the  data  given  the  type  of 
signal  detected  and  the  quality  of  the  signal. 

In  the  following  discussion,  rules  for  actioning  the  alert  data 
are  outlined  and  the  impact  of  using  this  approach  is  analyzed  through  the 
application  of  these  rules  to  the  380  SAR  incidents  which  formed  the  study 
data  base. 


Table  12(a)  categorizes  the  first  detection  for  the  380  SAR 
incidents  by  signal  type  according  to  whether  the  data  quality  was  observed 
|  to  be  good,  mediocre  or  poor.  These  data  are  presented  in  terms  of  whether 

the  transmission  was  seen  once  or  on  subsequent  passes. 

Firstly,  it  is  agreed  that  all  data  categorized  as  good  or 
mediocre  is  actionable  immediately.  Referring  to  Table  8  and  Figure  12, 
more  than  50%  of  these  data  should  exhibit  errors  of  estimation  of  less 
than  20  kilometres.  This  accounts  for  58%  of  the  SAR  cases.  It  is  further 
suggested  that  an  additional  47  cases,  or  12%,  warrant  immediate  SAR 
action.  These  are  the  poor  quality  E  and  I  type  signals.  The  signal  type 
categorization  conveys  the  fact  that  a  real  transmission  has  been  detected 
while  the  classification  of  the  quality  suggests  the  error  in  location 
estimation  is  large.  Therefore,  the  specific  SAR  response  would  be 
qualified.  The  data  is  sufficient  to  warrant  communication  checks  but  not 
of  good  enough  quality  to  immediately  consider  resource  expenditure.  In 
summary,  using  the  SAR  incident  data  to  establish  order  of  magnitude 
estimates,  70%  of  the  380  incidents  can  be  actioned  on  the  first  detection, 
58%  without  qualification  and  an  additional  12%  with  the  qualification  that 
the  location  estimate  is  quite  poor. 

The  remaining  30%  of  the  detections  require  additional 
information  before  any  SAR  action  can  be  taken.  These  are  the  U  type  of 
signals  of  poor  quality.  As  is  illustrated  in  Table  12(a)  24  incidents 
involving  a  single  Canadian  SARSAT  detection  fall  within  this  category. 
These  24  incidents  or  6%  of  the  cases  can  not  be  actioned  without  the  aid 
of  external  information,  e.g.  from  conventional  SAR  detection  means  or  from 
USA  SARSAT  sources. 


TABLE  12(a) 

SAR  CASES  -  FIRST  DETECTION  CATEGORIZATION 


SINGLE  DETECTION 
SIGNAL  TYPE 

DATA 

CATEGORY  U  E  I 

G  14  22  2 

M  13  4  0 

P  24  8  2 

51  34  4 


MULTIPLE  DETECTION 
SIGNAL  TYPE 


u 

E 

I 

TOTALS 

43 

62 

8 

151 

32 

18 

1 

68 

90 

36 

1 

161 

165 

116 

10 

380 

TABLE  12(b) 

SECOND  DETECTION 

ON  FIRST  DETECTION,  SIGNAL  TYPE  U  AND  CLASS  P 


DATA  SIGNAL  TYPE 


CATEGORY 

U 

E 

I 

TOTALS 

G 

12 

27 

0 

39 

M 

6 

8 

0 

14 

P 

31 

4 

2 

37 

TOTALS 

49 

39 

2 

90 

Table  12(b)  considers  the  remaining  90  incidents  or  24%  of  the 
cases.  These  are  the  U  type  signals  known  to  be  detected  on  a  subsequent 
satellite  pass.  On  the  second  detection,  59  of  the  events  now  meet  the 
standard  necessary  for  actioning.  Only  31  incidents  or  8%  of  the  cases 
fall  into  the  category  of  a  U  type  signal  of  poor  quality  on  the  first 
detection  and  remaining  that  way  through  the  second  detection.  All  that 
can  be  said  about  these  incidents  is  that  they  are  confirmed  events,  but 
that  the  ELT/EPIRB  is  of  such  poor  quality,  SARSAT  can  not  precisely  locate 
it. 


Figure  18 (a-b )  illustrates  the  ambiguity  resolution  capability  of 
SARSAT  for  the  alert  categorization  scheme  based  on  the  incident  data. 
Ambiguity  resolution  is  plotted  for  PROBS,  (based  on  standard  deviation) 
and  for  PROBT.  The  latter  is  derived  from  the  LUT  Trend  Factor.  Table  13 
contains  the  distribution  percentile  by  category  for  PROBS  and  PROBT  equal 
to  50.  The  percentile  greater  than  50  indicates  the  probability  of 
ambiguity  resolution. 
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TABLE  13 


AMBIGUITY  PROBABILITY  RESOLUTION 
BY  ALERT  CATEGORIZATION 


DATA  CATEGORY 
GOOD 

MEDIOCRE 

POOR 


PROBS 

PERCENT  >,  50 


PROBT 

PERCENT  50 


It  is  apparent  that  good  ambiguity  resolution  is  available  with 
good  and  mediocre  data.  The  PROBT  form  of  ambiguity  resolution  tends  to 
enhance  the  resolution  scale.  There  is  very  little  capability  to  resolve 
ambiguity  when  the  alert  data  is  categorized  as  poor. 

In  summary,  it  is  proposed  that  procedures  can  be  established  to 
action  SARSAT  alert  data  based  on  available  data  qualifiers.  Using  the  SAR 
incidents  which  occurred  during  the  SARSAT  D&E  as  a  basis  for  the 
development  of  these  procedures,  it  is  suggested  that  the  following 
approaches  seem  reasonable.  All  data  categorized  as  good  or  mediocre  are 
actionable  upon  receipt.  Furthermore,  the  probability  is  relatively  high 
that  ambiguity  can  be  resolved  on  the  first  pass.  It  should  be  recognized 
that  E  and  I  type  transmissions  of  poor  quality  are  valid  transmissions  and 
warrant  attention.  However  there  is  only  a  small  probability  that,  in  this 
case,  ambiguity  can  be  resolved  on  the  first  pass.  About  30%  of  the  SAR 
incident  alert  data  must  await  further  information.  They  are  not  of 
sufficient  quality  to  warrant  action  without  collaborating  information.  In 
the  case  of  the  SAR  incidents,  SARSAT  provided  this  information  for  one 
half  of  these  incidents  on  a  subsequent  pass.  Of  the  remaining  incidents, 
one  half  were  not  detected  again  and  the  other  half  continued  to 
demonstrate  poor  quality.  In  total,  85%  of  the  SAR  incidents  in  the  study 
data  base  were  resolvable  by  the  second  detection. 


6.0  SUMMARY  COMMENTS  AND  RECOMMENDATIONS 


It  is  concluded  that  SARSAT  system  performance  can  be  improved 
significantly  if  the  SAR  users  are  given  more  support  and  guidance  on  how 
to  Interpret  beacon  alert  data.  Alert  data  categorization  schemes  can  be 
developed  from  existing  LUT  parameter  data  which  could  facilitate  this 
interpretation. 


An  analysis  of  the  SAR  incidents  which  occurred  during  the  SARSAT 
D4E  has  demonstrated  the  capabilities  of  one  such  categorization  scheme. 
It  seems  to  work  quite  well.  In  hindsight,  it  is  viewed  as  an 

intuitatively  obvious  categorization  and  hence  is  probably  quite  a 
conservative  approach. 

It  should  be  noted  that  the  approaches  suggested  are  based  on  a 
statistical  analysis  of  highly  variable  data.  The  observed  operational 
ELT/EPIRB  performance  is  far  from  good.  Therefore,  one  can  expect 
instances  where  the  data  categorization  scheme  will  work  much  better  than 
suggested  but  also  there  will  be  instances  where  it  will  perform  quite 
poorly.  These  conclusions  necessarily  do  not  take  into  account  the 
positive  support  that  is  available  from  conventional  SAR  sources. 

In  conclusion,  it  is  recommended  that  the  SARSAT  alert  data 
categorization  scheme  developed  as  a  result  of  the  analysis  of  the  SAR 
incidents  which  occurred  during  the  SARSAT  D&E,  be  considered  for 
implementation  in  future  Canadian  SARSAT  facilities. 
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